Toward Patient-Centered Drug Approval for Treatment of Rare Diseases.

OBJECTIVES: This commentary seeks to improve the design and analysis of trials undertaken to obtain approval of drugs for treatment of rare diseases. METHODS: Methodological analysis reveals that use of hypothesis testing in the Food and Drug Administration drug approval process is harmful. Conventional asymmetric error probabilities bias the approval process against approval of new drugs. Hypothesis testing is inattentive to the relative magnitudes of losses to patient welfare when types 1 and 2 errors occur. Requiring the sample size to be large enough to guarantee the specified statistical power particularly inhibits the development of new drugs for treating rare diseases. Rarity of a disease makes it difficult to enroll the number of trial subjects needed to meet the statistical power standards for drug approval. RESULTS: Use of statistical decision theory in drug approval would overcome these serious deficiencies of hypothesis testing. Sample size would remain relevant to drug approval, but the criterion used to evaluate sample size would change. Rather than judging sample size by statistical power, the Food and Drug Administration could require a sample to be large enough to provide a specified nearness to optimality of the approval decision. CONCLUSIONS: Using nearness to optimality to set sample size and making approval decisions to minimize distance from optimality would particularly benefit the evaluation of drugs for treatment of rare diseases. It would enable a dramatic reduction in sample size relative to current norms, without compromising the clinical informativeness of trials.

Increasing prevalence of cirrhosis among insured adults in the United States, 2012-2018.

BACKGROUND: Liver cirrhosis is a chronic disease that is known as a "silent killer" and its true prevalence is difficult to describe. It is imperative to accurately characterize the prevalence of cirrhosis because of its increasing healthcare burden. METHODS: In this retrospective cohort study, trends in cirrhosis prevalence were evaluated using administrative data from one of the largest national health insurance providers in the US. (2011-2018). Enrolled adult (≥18-years-old) patients with cirrhosis defined by ICD-9 and ICD-10 were included in the study. The primary outcome measured in the study was the prevalence of cirrhosis 2011-2018. RESULTS: Among the 371,482 patients with cirrhosis, the mean age was 62.2 (±13.7) years; 53.3% had commercial insurance and 46.4% had Medicare Advantage. The most frequent cirrhosis etiologies were alcohol-related (26.0%), NASH (20.9%) and HCV (20.0%). Mean time of follow-up was 725 (±732.3) days. The observed cirrhosis prevalence was 0.71% in 2018, a 2-fold increase from 2012 (0.34%). The highest prevalence observed was among patients with Medicare Advantage insurance (1.67%) in 2018. Prevalence increased in each US. state, with Southern states having the most rapid rise (2.3-fold). The most significant increases were observed in patients with NASH (3.9-fold) and alcohol-related (2-fold) cirrhosis. CONCLUSION: Between 2012-2018, the prevalence of liver cirrhosis doubled among insured patients. Alcohol-related and NASH cirrhosis were the most significant contributors to this increase. Patients living in the South, and those insured by Medicare Advantage also have disproportionately higher prevalence of cirrhosis. Public health interventions are important to mitigate this concerning trajectory of strain to the health system.

Using measures of race to make clinical predictions: Decision making, patient health, and fairness.

The use of race measures in clinical prediction models is contentious. We seek to inform the discourse by evaluating the inclusion of race in probabilistic predictions of illness that support clinical decision making. Adopting a static utilitarian framework to formalize social welfare, we show that patients of all races benefit when clinical decisions are jointly guided by patient race and other observable covariates. Similar conclusions emerge when the model is extended to a two-period setting where prevention activities target systemic drivers of disease. We also discuss non-utilitarian concepts that have been proposed to guide allocation of health care resources.

Partial Identification of Personalized Treatment Response with Trial-reported Analyses of Binary Subgroups.

Medical journals have adhered to a reporting practice that seriously limits the usefulness of published trial findings. Medical decision makers commonly observe many patient covariates and seek to use this information to personalize treatment choices. Yet standard summaries of trial findings only partition subjects into broad subgroups, typically binary categories. Given this reporting practice, we study the problem of inference on long mean treatment outcomes E[y(t)|x], where t is a treatment, y(t) is a treatment outcome, and the covariate vector x has length K, each component being a binary variable. The available data are estimates of {E[y(t)|x k = 0], E[y(t)|x k = 1], P(x k )}, k = 1,..., K reported in journal articles. We show that reported trial findings partially identify {E[y(t)|x], P(x)}. Illustrative computations demonstrate that the summaries of trial findings in journal articles may imply only wide bounds on long mean outcomes. One can realistically tighten inferences if one can combine reported trial findings with credible assumptions having identifying power, such as bounded-variation assumptions.

Navigating Uncertainty in Immunotherapy Regimens.

Uncertainty in Immunotherapy RegimensIn this Patient Platform, Charles Manski, Ph.D., shares his experience in navigating uncertainty from the perspective of a patient who has had to cope with a challenging clinical problem and an academic whose research aims to improve the evidence and methods used to inform patient care.

Tail and Center Rounding of Probabilistic Expectations in the Health and Retirement Study.

We study rounding of numerical expectations in the Health and Retirement Study (HRS) between 2002 and 2014. We document that respondent-specific rounding patterns across questions in individual waves are quite stable across waves. We discover a tendency by about half of the respondents to provide more refined responses in the tails of the 0-100 scale than the center. In contrast, only about five percent of the respondents give more refined responses in the center than the tails. We find that respondents tend to report the values 25 and 75 more frequently than other values ending in 5. We also find that rounding practices vary somewhat across question domains and respondent characteristics. We propose an inferential approach that assumes stability of response tendencies across questions and waves to infer person-specific rounding in each question domain and scale segment and that replaces each point-response with an interval representing the range of possible values of the true latent belief. Using expectations from the 2016 wave of the HRS, we validate our approach. To demonstrate the consequences of rounding on inference, we compare best-predictor estimates from face-value expectations with those implied by our intervals.

Patient-centered appraisal of race-free clinical risk assessment.

Until recently, there has been a consensus that clinicians seeking to assess patient risks of illness should condition risk assessments on all observed patient covariates with predictive power. The broad idea is that knowing more about patients enables more accurate predictions of their health risks and, hence, better clinical decisions. This consensus has recently unraveled with respect to a specific covariate, namely race. There have been increasing calls for race-free risk assessment, arguing that using race to predict health risks contributes to racial disparities and inequities in health care. In some medical fields, leading institutions have recommended race-free risk assessment. An important open question is how race-free risk assessment would affect the quality of clinical decisions. Considering the matter from the patient-centered perspective of medical economics yields a disturbing conclusion: Race-free risk assessment would harm patients of all races.

Balancing data privacy and usability in the federal statistical system.

The federal statistical system is experiencing competing pressures for change. On the one hand, for confidentiality reasons, much socially valuable data currently held by federal agencies is either not made available to researchers at all or only made available under onerous conditions. On the other hand, agencies which release public databases face new challenges in protecting the privacy of the subjects in those databases, which leads them to consider releasing fewer data or masking the data in ways that will reduce their accuracy. In this essay, we argue that the discussion has not given proper consideration to the reduced social benefits of data availability and their usability relative to the value of increased levels of privacy protection. A more balanced benefit-cost framework should be used to assess these trade-offs. We express concerns both with synthetic data methods for disclosure limitation, which will reduce the types of research that can be reliably conducted in unknown ways, and with differential privacy criteria that use what we argue is an inappropriate measure of disclosure risk. We recommend that the measure of disclosure risk used to assess all disclosure protection methods focus on what we believe is the risk that individuals should care about, that more study of the impact of differential privacy criteria and synthetic data methods on data usability for research be conducted before either is put into widespread use, and that more research be conducted on alternative methods of disclosure risk reduction that better balance benefits and costs.

Precise or Imprecise Probabilities? Evidence from Survey Response Related to Late-Onset Dementia.

We elicit numerical expectations for late-onset dementia and long-term-care (LTC) outcomes in the US Health and Retirement Study. We provide the first empirical evidence on dementia-risk perceptions among dementia-free older Americans and establish important patterns regarding imprecision of subjective probabilities. Our elicitation distinguishes between precise and imprecise probabilities, while accounting for rounding of reports. Imprecise-probability respondents quantify imprecision using probability intervals. Nearly half of respondents hold imprecise dementia and LTC probabilities, while almost a third of precise-probability respondents round their reports. These proportions decrease substantially when LTC expectations are conditioned on hypothetical knowledge of the dementia state. Among rounding and imprecise-probability respondents, our elicitation yields two measures: an initial rounded or approximated response and a post-probe response, which we interpret as the respondent's true point or interval probability. We study the mapping between the two measures and find that respondents initially tend to over-report small probabilities and under-report large probabilities. Using a specific framework for study of LTC insurance choice with uncertain dementia state, we illustrate the dangers of ignoring imprecise or rounded probabilities for modeling and prediction of insurance demand.

Definition of a critical bleed in patients with immune thrombocytopenia: Communication from the ISTH SSC Subcommittee on Platelet Immunology.

BACKGROUND: Immune thrombocytopenia (ITP) is an autoimmune disease characterized by low platelet counts and increased risk of bleeding. In preparation for an upcoming guideline, the ITP Emergency Management Guideline Panel, including clinical experts in hematology, emergency medicine, research methodology, and patient representatives, identified the need for a standardized definition of a critical ITP bleed. The goal of the definition was to distinguish critical bleeds from bleeds that may not require urgent treatment, typically in the context of severe thrombocytopenia. METHODS: The panel met in person and virtually to achieve consensus on the criteria for critical bleeding events among patients with ITP. Existing ITP bleeding scores and published definitions of major bleeds in patients receiving anticoagulation informed the definition of a critical ITP bleed. The Platelet Immunology Scientific Standardization Committee (SSC) of the International Society on Thrombosis and Haemostasis endorsed the definition. RESULTS: A critical ITP bleed was defined as: (a) a bleed in a critical anatomical site including intracranial, intraspinal, intraocular, retroperitoneal, pericardial, or intramuscular with compartment syndrome; or (2) an ongoing bleed that results in hemodynamic instability or respiratory compromise. CONCLUSION: The definition of a critical ITP bleed was developed by the ITP Emergency Management Guideline Panel and endorsed by the Platelet Immunology SSC. It incorporates both anatomic and physiologic risk and pertains to patients with confirmed or suspected ITP who typically have severe thrombocytopenia (platelet count below 20 × 109 /L).

Embracing Uncertainty: The Value of Partial Identification in Public Health and Clinical Research.

INTRODUCTION: This paper describes the methodology of partial identification and its applicability to empirical research in preventive medicine and public health. METHODS: The authors summarize findings from the methodologic literature on partial identification. The analysis was conducted in 2020-2021. RESULTS: The applicability of partial identification methods is demonstrated using 3 empirical examples drawn from published literature. CONCLUSIONS: Partial identification methods are likely to be of considerable interest to clinicians and others engaged in preventive medicine and public health research.

Statistical Decision Properties of Imprecise Trials Assessing Coronavirus Disease 2019 (COVID-19) Drugs.

OBJECTIVES: Researchers studying treatment of coronavirus disease 2019 (COVID-19) have reported findings of randomized trials comparing standard care with care augmented by experimental drugs. Many trials have small sample sizes, so estimates of treatment effects are imprecise. Hence, clinicians may find it difficult to decide when to treat patients with experimental drugs. A conventional practice when comparing standard care and an innovation is to choose the innovation only if the estimated treatment effect is positive and statistically significant. This practice defers to standard care as the status quo. We study treatment choice from the perspective of statistical decision theory, which considers treatment options symmetrically when assessing trial findings. METHODS: We use the concept of near-optimality to evaluate criteria for treatment choice. This concept jointly considers the probability and magnitude of decision errors. An appealing criterion from this perspective is the empirical success rule, which chooses the treatment with the highest observed average patient outcome in the trial. RESULTS: Considering the design of some COVID-19 trials, we show that the empirical success rule yields treatment choices that are much closer to optimal than those generated by prevailing decision criteria based on hypothesis tests. CONCLUSION: Using trial findings to make near-optimal treatment choices rather than perform hypothesis tests should improve clinical decision making.

Addressing partial identification in climate modeling and policy analysis.

Numerical simulations of the global climate system provide inputs to integrated assessment modeling for estimating the impacts of greenhouse gas mitigation and other policies to address global climate change. While essential tools for this purpose, computational climate models are subject to considerable uncertainty, including intermodel "structural" uncertainty. Structural uncertainty analysis has emphasized simple or weighted averaging of the outputs of multimodel ensembles, sometimes with subjective Bayesian assignment of probabilities across models. However, choosing appropriate weights is problematic. To use climate simulations in integrated assessment, we propose, instead, framing climate model uncertainty as a problem of partial identification, or "deep" uncertainty. This terminology refers to situations in which the underlying mechanisms, dynamics, or laws governing a system are not completely known and cannot be credibly modeled definitively even in the absence of data limitations in a statistical sense. We propose the min-max regret (MMR) decision criterion to account for deep climate uncertainty in integrated assessment without weighting climate model forecasts. We develop a theoretical framework for cost-benefit analysis of climate policy based on MMR, and apply it computationally with a simple integrated assessment model. We suggest avenues for further research.

Bounding the Accuracy of Diagnostic Tests, With Application to COVID-19 Antibody Tests.

Tests used to diagnose illness commonly have imperfect accuracy, with some false-positive and negative results. For risk assessment and clinical decisions, predictive values are of interest. Positive predictive value (PPV) is the chance that a member of a relevant population who tests positive has been ill. Negative predictive value (NPV) is the chance that someone who tests negative has not been ill. The medical literature regularly reports sensitivity and specificity. Sensitivity is the chance that an ill person receives a positive test result. Specificity is the chance that a nonill person receives a negative result. Knowledge of sensitivity and specificity enables one to predict the test result given a person's illness status. These predictions are not directly relevant to patient care but, given knowledge of sensitivity and specificity, PPV and NPV can be derived if one knows the prevalence of the disease, the population rate of illness. There is considerable uncertainty about the prevalence of some diseases, a notable case being COVID-19. This paper addresses the problem of identification of PPV and NPV given knowledge of sensitivity and specificity and given bounds on prevalence. I explain the problem and show how to bound PPV and NPV as well as the risk ratio and difference, which are functions thereof. I apply the findings to COVID-19 antibody tests. I question the realism of supposing that sensitivity and specificity are known.

Estimating the COVID-19 infection rate: Anatomy of an inference problem.

As a consequence of missing data on tests for infection and imperfect accuracy of tests, reported rates of cumulative population infection by the SARS CoV-2 virus are lower than actual rates of infection. Hence, reported rates of severe illness conditional on infection are higher than actual rates. Understanding the time path of the COVID-19 pandemic has been hampered by the absence of bounds on infection rates that are credible and informative. This paper explains the logical problem of bounding these rates and reports illustrative findings, using data from Illinois, New York, and Italy. We combine the data with assumptions on the infection rate in the untested population and on the accuracy of the tests that appear credible in the current context. We find that the infection rate might be substantially higher than reported. We also find that, assuming accurate reporting of deaths, the infection fatality rates in Illinois, New York, and Italy are substantially lower than reported.

Toward Credible Patient-centered Meta-analysis.

Meta-analysis is widely used to combine the findings of multiple disparate studies of health risks or treatment response. Meta-analysis often uses a random-effects model to express heterogeneity across studies. The model interprets a weighted average of study-specific estimates as an estimate of a mean parameter across a hypothetical population of studies. The relevance of this methodology to patient care is not evident. Clinicians need to assess risks and choose treatments for populations of patients, not for populations of studies. This article draws on econometric research on partial identification to propose principles for patient-centered meta-analysis. One specifies a patient prediction of concern and determines what each available study reveals. Given common imperfections in internal and external validity, studies typically yield credible set-valued rather than point predictions. Thus, a study may enable one to conclude that a probability of disease, or mean treatment response, lies within a range of possibilities. Patient-centered meta-analysis would combine the findings of multiple studies by computing the intersection of the set-valued predictions that they yield.

Tuberculosis diagnosis and treatment under uncertainty.

In 2017, 1.6 million people worldwide died from tuberculosis (TB). A new TB diagnostic test-Xpert MTB/RIF from Cepheid-was endorsed by the World Health Organization in 2010. Trials demonstrated that Xpert is faster and has greater sensitivity and specificity than smear microscopy-the most common sputum-based diagnostic test. However, subsequent trials found no impact of introducing Xpert on morbidity and mortality. We present a decision-theoretic model of how a clinician might decide whether to order Xpert or other tests for TB, and whether to treat a patient, with or without test results. Our first result characterizes the conditions under which it is optimal to perform empirical treatment; that is, treatment without diagnostic testing. We then examine the implications for decision making of partial knowledge of TB prevalence or test accuracy. This partial knowledge generates ambiguity, also known as deep uncertainty, about the best testing and treatment policy. In the presence of such ambiguity, we show the usefulness of diversification of testing and treatment.